Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add android-ndk r21d #3004

Merged
merged 10 commits into from
Jan 4, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions recipes/android-ndk/all/cmake-wrapper
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/usr/bin/env bash

echo "Calling CMake via Android-NDK wrapper"

ALL_ARGS=$@

BUILD=no
for i in "$@":
do
case $i in
--build)
BUILD=yes
;;
*)
;;
esac
done

# https://gitlab.kitware.com/cmake/cmake/issues/18739 Android: NDK r19 support
# https://github.com/conan-io/conan/issues/2402 Android and CONAN_LIBCXX do not play well together
# https://github.com/conan-io/conan/issues/4537 Pass ANDROID_ABI & ANDROID_NDK variables to cmake
# https://github.com/conan-io/conan/issues/4629 set(CMAKE_FIND_ROOT_PATH_MODE_* ONLY) when cross-compiling, for Android at least

if [ $BUILD == "yes" ]; then
cmake "$@"
else
cmake \
-DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=$CMAKE_FIND_ROOT_PATH_MODE_PROGRAM \
-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=$CMAKE_FIND_ROOT_PATH_MODE_LIBRARY \
-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=$CMAKE_FIND_ROOT_PATH_MODE_INCLUDE \
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=$CMAKE_FIND_ROOT_PATH_MODE_PACKAGE \
-DANDROID_STL=$ANDROID_STL \
-DANDROID_NDK=$ANDROID_NDK \
-DANDROID_ABI=$ANDROID_ABI \
-DANDROID_PLATFORM=$ANDROID_PLATFORM \
-DANDROID_TOOLCHAIN=$ANDROID_TOOLCHAIN \
-DANDROID_NATIVE_API_LEVEL=$ANDROID_NATIVE_API_LEVEL \
"$@"
fi
35 changes: 35 additions & 0 deletions recipes/android-ndk/all/cmake-wrapper.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
@echo off

set BUILD=no

for %%i in (%*) do (

if %%i==--build set BUILD=yes
)


if %BUILD%==yes (

cmake %*

) else (

REM https://gitlab.kitware.com/cmake/cmake/issues/18739 Android: NDK r19 support
REM https://github.com/conan-io/conan/issues/2402 Android and CONAN_LIBCXX do not play well together
REM https://github.com/conan-io/conan/issues/4537 Pass ANDROID_ABI & ANDROID_NDK variables to cmake
REM https://github.com/conan-io/conan/issues/4629 set(CMAKE_FIND_ROOT_PATH_MODE_* ONLY) when cross-compiling, for Android at least

cmake ^
-DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=%CMAKE_FIND_ROOT_PATH_MODE_PROGRAM% ^
-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=%CMAKE_FIND_ROOT_PATH_MODE_LIBRARY% ^
-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=%CMAKE_FIND_ROOT_PATH_MODE_INCLUDE% ^
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=%CMAKE_FIND_ROOT_PATH_MODE_PACKAGE% ^
-DANDROID_STL=%ANDROID_STL% ^
-DANDROID_NDK=%ANDROID_NDK% ^
-DANDROID_ABI=%ANDROID_ABI% ^
-DANDROID_PLATFORM=%ANDROID_PLATFORM% ^
-DANDROID_TOOLCHAIN=%ANDROID_TOOLCHAIN% ^
-DANDROID_NATIVE_API_LEVEL=%ANDROID_NATIVE_API_LEVEL% ^
%*

)
11 changes: 11 additions & 0 deletions recipes/android-ndk/all/conandata.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
sources:
"r21d":
"Windows":
url: https://dl.google.com/android/repository/android-ndk-r21d-windows-x86_64.zip
sha256: 3f4257fc59520ca2366b172c6f7dcff2df06436b668928b0fcd0bab1f24b8b8b
"Linux":
url: https://dl.google.com/android/repository/android-ndk-r21d-linux-x86_64.zip
sha256: 3f4257fc59520ca2366b172c6f7dcff2df06436b668928b0fcd0bab1f24b8b8b
"Macos":
url: https://dl.google.com/android/repository/android-ndk-r21d-darwin-x86_64.zip
sha256: 5851115c6fc4cce26bc320295b52da240665d7ff89bda2f5d5af1887582f5c48
212 changes: 212 additions & 0 deletions recipes/android-ndk/all/conanfile.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
from conans import ConanFile, tools
from conans.errors import ConanInvalidConfiguration
import os


class AndroidNDKInstallerConan(ConanFile):
name = "android-ndk"
description = "The Android NDK is a toolset that lets you implement parts of your app in " \
"native code, using languages such as C and C++"
url = "https://github.com/conan-io/conan-center-index"
homepage = "https://developer.android.com/ndk/"
topics = ("NDK", "android", "toolchain", "compiler")
license = "Apache-2.0"
short_paths = True
no_copy_source = True
exports_sources = ["cmake-wrapper.cmd", "cmake-wrapper"]

settings = {"os": ["Windows", "Linux", "Macos"],
a4z marked this conversation as resolved.
Show resolved Hide resolved
"arch": ["x86_64"]}

@property
def _source_subfolder(self):
return "source_subfolder"

def configure(self):
if self.settings.arch != 'x86_64':
raise ConanInvalidConfiguration("No binaries available for other than 'x86_64' architectures")

def source(self):
tarballs = self.conan_data["sources"][self.version]
tools.get(**tarballs[str(self.settings.os)])
extracted_dir = self.name + "-" + self.version
os.rename(extracted_dir, self._source_subfolder)
Copy link
Contributor

@madebr madebr Sep 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the downloading of the binaries should move to build,
because the following will fail:

conan create android-ndk/r21d@ -s os=Windows
conan create android-ndk/r21d@ -s os=Linux -ks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not understand
If you want to download this for a operating system you are not on, this package will not make any sense

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would make it possible for a devops guy to create android-ndk packages on his Linux machine for all platforms.
This is possible now too, but there is a possibility that, by executing the commands in my previous comment, a wrong package is generated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a good point I never thought about because I have all 3 OSes just available,
but I sometimes found it annoying to change the computer
So I will investigate in this, thanks!


def build(self):
pass # no build, but please also no warnings

# from here on, everything is assumed to run in 2 profile mode, when using the ndk as a build requirement

@property
def _platform(self):
return {"Windows": "windows",
"Macos": "darwin",
"Linux": "linux"}.get(str(self.settings_build.os))

@property
def _android_abi(self):
return {"x86": "x86",
"x86_64": "x86_64",
"armv7": "armeabi-v7a",
"armv8": "arm64-v8a"}.get(str(self.settings_target.arch))

@property
def _llvm_triplet(self):
arch = {'armv7': 'arm',
'armv8': 'aarch64',
'x86': 'i686',
'x86_64': 'x86_64'}.get(str(self.settings_target.arch))
abi = 'androideabi' if self.settings_target.arch == 'armv7' else 'android'
return '%s-linux-%s' % (arch, abi)

@property
def _clang_triplet(self):
arch = {'armv7': 'armv7a',
'armv8': 'aarch64',
'x86': 'i686',
'x86_64': 'x86_64'}.get(str(self.settings_target.arch))
abi = 'androideabi' if self.settings_target.arch == 'armv7' else 'android'
return '%s-linux-%s' % (arch, abi)
a4z marked this conversation as resolved.
Show resolved Hide resolved

def _fix_permissions(self):
if os.name != 'posix':
return
for root, _, files in os.walk(self.package_folder):
for filename in files:
filename = os.path.join(root, filename)
with open(filename, 'rb') as f:
sig = f.read(4)
if type(sig) is str:
sig = [ord(s) for s in sig]
else:
sig = [s for s in sig]
if len(sig) > 2 and sig[0] == 0x23 and sig[1] == 0x21:
self.output.info('chmod on script file: "%s"' % filename)
self._chmod_plus_x(filename)
elif sig == [0x7F, 0x45, 0x4C, 0x46]:
self.output.info('chmod on ELF file: "%s"' % filename)
self._chmod_plus_x(filename)
elif sig == [0xCA, 0xFE, 0xBA, 0xBE] or \
sig == [0xBE, 0xBA, 0xFE, 0xCA] or \
sig == [0xFE, 0xED, 0xFA, 0xCF] or \
sig == [0xCF, 0xFA, 0xED, 0xFE] or \
sig == [0xFE, 0xEF, 0xFA, 0xCE] or \
sig == [0xCE, 0xFA, 0xED, 0xFE]:
self.output.info('chmod on Mach-O file: "%s"' % filename)
self._chmod_plus_x(filename)

def package(self):
self.copy(pattern="*", dst=".", src=self._source_subfolder, keep_path=True, symlinks=True)
self.copy(pattern="*", dst="bin", src=self._source_subfolder, keep_path=True, symlinks=True)
self.copy(pattern="*NOTICE", dst="licenses", src=self._source_subfolder)
self.copy(pattern="*NOTICE.toolchain", dst="licenses", src=self._source_subfolder)
self.copy("cmake-wrapper.cmd")
self.copy("cmake-wrapper")
self._fix_permissions()

@property
def _host(self):
return self._platform + "-x86_64"

@property
def _ndk_root(self):
return os.path.join(self.package_folder, "toolchains", "llvm", "prebuilt", self._host)

def _tool_name(self, tool):
if 'clang' in tool:
suffix = '.cmd' if self.settings_build.os == 'Windows' else ''
return '%s%s-%s%s' % (self._clang_triplet, self.settings_target.os.api_level, tool, suffix)
else:
suffix = '.exe' if self.settings_build.os == 'Windows' else ''
return '%s-%s%s' % (self._llvm_triplet, tool, suffix)

def _define_tool_var(self, name, value):
ndk_bin = os.path.join(self._ndk_root, 'bin')
path = os.path.join(ndk_bin, self._tool_name(value))
self.output.info('Creating %s environment variable: %s' % (name, path))
return path

# def package_id(self):
# self.info.include_build_settings()
# del self.info.settings.arch
# del self.info.settings.os.api_level

@staticmethod
def _chmod_plus_x(filename):
if os.name == 'posix':
os.chmod(filename, os.stat(filename).st_mode | 0o111)

def package_info(self):
Copy link
Contributor

@madebr madebr Sep 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this package have a toolchain method?
See https://docs.conan.io/en/latest/creating_packages/toolchains/cmake.html

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs further investigation since I never used that methode, but a good point!

also, there is actually a toolchain file included in, /build/cmake/android.toolchain.cmake
need to check if this could simplify the cmake wrapper

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also interested in knowing if this package will be compatible with the new toolchain features.
For instance here https://docs.conan.io/en/latest/integrations/cross_platform/android.html#use-built-in-conan-toolchain they use a profile:host with android_ndk_installer/r20@bincrafters/stable and conan_toolchain.cmake gets filled with all the information.

Is the current package a drop-in replacement for the bincrafters recipe?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hopobcn , this NDK package makes much more, it allows also building packages that do not use CMake, like openssl or curl

Why creating an own toolchain if you can get one from the NDK, that is for sure the most complete one?
If you create your toolchain file, it should just include that one from the NDK anyway

It is not a drop in replacement, because it works with the (not so) new 2 profile approach, the bincrafter one used the one profile way with os_buid and os_arch
The future is the 2 profile way.

# test shall pass, so this runs also in the build as build requirement context
# ndk-build: https://developer.android.com/ndk/guides/ndk-build
self.env_info.PATH.append(os.path.join(self.package_folder, 'bin'))

# this is not enough, I can kill that .....
if not hasattr(self, 'settings_target'):
return

# interestingly I can reach that with
# conan test --profile:build nsdk-default --profile:host default /Users/a4z/elux/conan/myrecipes/android-ndk/all/test_package android-ndk/r21d@
if self.settings_target is None:
return

# And if we are not building for Android, why bother at all
if not self.settings_target.os == "Android":
return

self.output.info('Creating NDK_ROOT environment variable: %s' % self._ndk_root)
self.env_info.NDK_ROOT = self._ndk_root

self.output.info('Creating ANDROID_NDK_HOME environment variable: %s' % self.package_folder)
self.env_info.ANDROID_NDK_HOME = self.package_folder

self.output.info('Creating CHOST environment variable: %s' % self._llvm_triplet)
self.env_info.CHOST = self._llvm_triplet

ndk_sysroot = os.path.join(self._ndk_root, 'sysroot')
self.output.info('Creating CONAN_CMAKE_FIND_ROOT_PATH environment variable: %s' % ndk_sysroot)
self.env_info.CONAN_CMAKE_FIND_ROOT_PATH = ndk_sysroot

self.output.info('Creating SYSROOT environment variable: %s' % ndk_sysroot)
self.env_info.SYSROOT = ndk_sysroot

self.output.info('Creating self.cpp_info.sysroot: %s' % ndk_sysroot)
self.cpp_info.sysroot = ndk_sysroot

self.output.info('Creating ANDROID_NATIVE_API_LEVEL environment variable: %s' % self.settings_target.os.api_level)
self.env_info.ANDROID_NATIVE_API_LEVEL = str(self.settings_target.os.api_level)

self._chmod_plus_x(os.path.join(self.package_folder, "cmake-wrapper"))
cmake_wrapper = "cmake-wrapper.cmd" if self.settings.os == "Windows" else "cmake-wrapper"
cmake_wrapper = os.path.join(self.package_folder, cmake_wrapper)
self.output.info('Creating CONAN_CMAKE_PROGRAM environment variable: %s' % cmake_wrapper)
self.env_info.CONAN_CMAKE_PROGRAM = cmake_wrapper

toolchain = os.path.join(self.package_folder, "build", "cmake", "android.toolchain.cmake")
self.output.info('Creating CONAN_CMAKE_TOOLCHAIN_FILE environment variable: %s' % toolchain)
self.env_info.CONAN_CMAKE_TOOLCHAIN_FILE = toolchain

self.env_info.CC = self._define_tool_var('CC', 'clang')
self.env_info.CXX = self._define_tool_var('CXX', 'clang++')
self.env_info.LD = self._define_tool_var('LD', 'ld')
self.env_info.AR = self._define_tool_var('AR', 'ar')
self.env_info.AS = self._define_tool_var('AS', 'as')
self.env_info.RANLIB = self._define_tool_var('RANLIB', 'ranlib')
self.env_info.STRIP = self._define_tool_var('STRIP', 'strip')
self.env_info.ADDR2LINE = self._define_tool_var('ADDR2LINE', 'addr2line')
self.env_info.NM = self._define_tool_var('NM', 'nm')
self.env_info.OBJCOPY = self._define_tool_var('OBJCOPY', 'objcopy')
self.env_info.OBJDUMP = self._define_tool_var('OBJDUMP', 'objdump')
self.env_info.READELF = self._define_tool_var('READELF', 'readelf')
self.env_info.ELFEDIT = self._define_tool_var('ELFEDIT', 'elfedit')

self.env_info.ANDROID_PLATFORM = "android-%s" % self.settings_target.os.api_level
self.env_info.ANDROID_TOOLCHAIN = "clang"
self.env_info.ANDROID_ABI = self._android_abi
libcxx_str = str(self.settings_target.compiler.libcxx)
self.env_info.ANDROID_STL = libcxx_str if libcxx_str.startswith('c++_') else 'c++_shared'

self.env_info.CMAKE_FIND_ROOT_PATH_MODE_PROGRAM = "BOTH"
self.env_info.CMAKE_FIND_ROOT_PATH_MODE_LIBRARY = "BOTH"
self.env_info.CMAKE_FIND_ROOT_PATH_MODE_INCLUDE = "BOTH"
self.env_info.CMAKE_FIND_ROOT_PATH_MODE_PACKAGE = "BOTH"
a4z marked this conversation as resolved.
Show resolved Hide resolved
13 changes: 13 additions & 0 deletions recipes/android-ndk/all/test_package/conanfile.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from conans import ConanFile, tools
import os


class TestPackgeConan(ConanFile):
settings = "os", "arch"

def build(self):
pass #nothing to do, not warnings please

def test(self):
if not tools.cross_building(self):
self.run("ndk-build --version", run_environment=True)
4 changes: 4 additions & 0 deletions recipes/android-ndk/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
versions:
"r21d":
folder: all