Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI build for Android #3

Merged
merged 27 commits into from
Jun 30, 2024
Merged

CI build for Android #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
b0580fc
Squash commits - Build for Android
ViliusSutkus89 Jun 15, 2024
a8405c0
[CI] Use host profile when exporting binary pkg
ViliusSutkus89 Jun 16, 2024
220e557
Remove conan graph, it generated way too much files
ViliusSutkus89 Jun 16, 2024
b3088ef
Login to artifactory only if there will be an actual upload
ViliusSutkus89 Jun 16, 2024
b2cba3f
Update build matrix
ViliusSutkus89 Jun 16, 2024
64f73e7
Exclude odrcore/1.0.0 from build matrix when building for Android
ViliusSutkus89 Jun 16, 2024
7d1a1bd
Use variables in conan profiles to set ANDROID_HOME
ViliusSutkus89 Jun 17, 2024
300187e
Use package/version and host profile as CI build job name
ViliusSutkus89 Jun 18, 2024
2ce3cb9
Rename package_version to package_reference
ViliusSutkus89 Jun 20, 2024
5d0d644
Add Android-21 and -23 conan profiles
ViliusSutkus89 Jun 20, 2024
7020959
List (and build in CI) newer versions first
ViliusSutkus89 Jun 20, 2024
92d6978
[2/2] Rename package_version to package_reference
ViliusSutkus89 Jun 20, 2024
5dc5afc
Add deep test workflow
ViliusSutkus89 Jun 20, 2024
6175876
Disable package exporter call in deep-test
ViliusSutkus89 Jun 20, 2024
71d8669
Update deep test filter
ViliusSutkus89 Jun 20, 2024
e0495be
Export all conan recipes to local cache
ViliusSutkus89 Jun 24, 2024
96075e5
Trigger build pipeline on generic push (this should also cover on PR)
ViliusSutkus89 Jun 24, 2024
74e2ad7
[2/2] Trigger build pipeline on generic push (this should also cover …
ViliusSutkus89 Jun 24, 2024
3c5a3a3
Disable ccache in CI. Build will not fit anyways
ViliusSutkus89 Jun 24, 2024
1f949b9
Merge deep-test into build pipeline
ViliusSutkus89 Jun 24, 2024
2da8205
Use ubuntu build profile instead of relying on default build profile
ViliusSutkus89 Jun 24, 2024
9db47be
Cache pip dependencies
ViliusSutkus89 Jun 24, 2024
befbf53
Run build pipeline on workflow_dispatch too
ViliusSutkus89 Jun 24, 2024
cb0cf52
Remove pip cache, turns out it's only for regular python projects wit…
ViliusSutkus89 Jun 24, 2024
a39d79b
Revert "Disable ccache in CI. Build will not fit anyways". I may have…
ViliusSutkus89 Jun 24, 2024
85ac402
Show ccache stats
ViliusSutkus89 Jun 24, 2024
37bb728
Update conan_export_all_packages to properly handle paths when starte…
ViliusSutkus89 Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-21-armv7
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "21" %}
{% set arch = "armv7" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
andiwand marked this conversation as resolved.
Show resolved Hide resolved
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
Comment on lines +26 to +31
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: can this be moved up to the other set section?

similar below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-21-armv8
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "21" %}
{% set arch = "armv8" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-21-x86
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "21" %}
{% set arch = "x86" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-21-x86_64
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "21" %}
{% set arch = "x86_64" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-23-armv7
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "23" %}
{% set arch = "armv7" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-23-armv8
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "23" %}
{% set arch = "armv8" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-23-x86
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "23" %}
{% set arch = "x86" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
35 changes: 35 additions & 0 deletions .github/config/ubuntu-22.04/conan/profiles/android-23-x86_64
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{% set android_home = os.getenv("ANDROID_HOME") %}
{% set ndk_version = "26.3.11579264" %}
{% set api_level = "23" %}
{% set arch = "x86_64" %}

[settings]
os=Android
os.api_level={{api_level}}
arch={{arch}}
build_type=RelWithDebInfo
compiler=clang
compiler.version=17
compiler.cppstd=20
compiler.libcxx=c++_static

[conf]
tools.android:ndk_path={{android_home}}/ndk/{{ndk_version}}

[buildenv]
# Cross compile toolchain evn vars are required to build
# libffi, libgettext, libiconv, libxml2 and other autotools packages for Android
# https://github.com/conan-io/conan/issues/16493
AR={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
AS={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-as
RANLIB={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
{% set cc = {
"armv7": "armv7a-linux-androideabi" + api_level + "-clang",
"armv8": "aarch64-linux-android" + api_level + "-clang",
"x86": "i686-linux-android" + api_level + "-clang",
"x86_64": "x86_64-linux-android" + api_level + "-clang",
}[arch] %}
CC={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}
CXX={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/{{cc}}++
LD={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/ld
STRIP={{android_home}}/ndk/{{ndk_version}}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
6 changes: 3 additions & 3 deletions ...ntu-22.04-clang-15/conan/profiles/default → ...onfig/ubuntu-22.04/conan/profiles/default
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
[settings]
os=Linux
arch=x86_64
build_type=Release
build_type=RelWithDebInfo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for changing this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain this?

"-s", "build_type=Release",
"-s", "&:build_type=RelWithDebInfo",
"-s", "odrcore/*:build_type=RelWithDebInfo",

I assume this means a request to build everything as Release, but only odr.core as RelWithDebInfo ? I assume that if we build it like that, there will be less debug info in dependencies. Most of pdf crashes happens not in odr.core, but somewhere deep in libpng, called by either fontforge or poppler, because pdf2htmlEX supplied the wrong argument somewhere. I assume Release instead of RelWithDebInfo for deep dependencies would give less info during crashes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes my intention here was to only have debug symbols for odr.core and rely on other packages to be stable. That paradigm might shift with pdf2htmlEX.

I think I was also facing some multi config issues with Android.

Ultimately this should not necessarily drive our decision here. I think Release and RelWithDebInfo is fine while Debug might be a bit too big.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug is completely unoptimized, let's leave RelWithDebInfo. We could also test if RelWithDebInfo actually produces binaries that are slower or larger than Release

compiler=clang
compiler.version=15
compiler.cppstd=17
compiler.cppstd=20
compiler.libcxx=libstdc++11
os=Linux

[conf]
tools.build:compiler_executables={'c': 'clang-15', 'cpp': 'clang++-15'}
1 change: 1 addition & 0 deletions .github/config/ubuntu-22.04/conan/profiles/ubuntu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
default
Loading